In modern processors adopting a virtual memory system architecture, a virtual memory address is translated into a physical memory address before the virtual memory address is used to fetch instruction or data contents from the physical memory. A translation process of the physical memory address typically involves a Translation Look-aside Buffer (TLB) lookup first. If there is a hit in the TLB, a translation result is returned. If there is a miss, the process proceeds to a “table walk” to acquire translation data. A table walk generally involves several memory accesses. Thus, TLB misses generally lead to a TLB miss penalty that can be large (e.g., many 100's of cycles). Larger virtual address ranges correspond to a larger TLB miss-penalty since larger virtual address ranges involve more levels of walk and more memory accesses.